Ultra-wide field-of-view scanning devices for depth sensing

ABSTRACT

A depth camera assembly for determining depth information for objects in a local area comprises a light generator, a camera and a controller. The light generator illuminates the local area with structured light in accordance with emission instructions from the controller. The light generator includes an illumination source, an acousto-optic deflector (AOD), and a liquid crystal device (LCD) with liquid crystal gratings (LCGs). The AOD functions as a dynamic diffraction grating that diffracts optical beams emitted from the illumination source to form diffracted scanning beams, based on emission instructions from the controller. Each LCG in the LCD is configured to further diffract light from the AOD to generate the structured light projected into the local area. The camera captures images of portions of the structured light reflected from objects in the local area. The controller determines depth information for the objects based on the captured images.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/960,045, filed Apr. 23, 2018, which claims benefit of U.S.Provisional Patent Application Ser. No. 62/513,286, filed May 31, 2017,which is hereby incorporated by reference in its entirety.

BACKGROUND

The present disclosure generally relates to depth sensing, andspecifically relates to ultra-wide field-of-view scanning devices forthree-dimensional (3D) depth sensing.

To achieve a compelling user experience for depth sensing when usinghead-mounted displays (HMDs) and near-eye displays (NEDs), it isimportant to create a dynamic and all solid-state light scanning devicewith both ultrafast scanning speed (e.g., MHz) and large field-of-view.Usually, there are tradeoffs between speed, field-of-view and real-timereconfigurable illumination characteristics. Typically, amicroelectromechanical system (MEM) having a mechanical-based mirrordevice can be used for scanning. However, the mechanical-based mirrordevice has stability issues and has a limited scanning speed. Inaddition, the mechanical-based mirror device is not reconfigurable inreal time applications.

Most depth sensing methods rely on active illumination and detection.The conventional methods for depth sensing involve mechanical scanningor fixed diffractive-optics pattern projection, using structured lightor time-of-flight techniques. Depth sensing based on time-of-flight usesa MEM with a mechanical-based mirror device (scanner) to send shortpulses into an object space. The depth sensing based on time-of-flightfurther uses a high speed detector to time-gate back scattered lightfrom the object to create high resolution depth maps. However, themechanical-based scanner performs inadequately in relation to scanningspeed, real-time reconfiguration and mechanical stability. The scanningspeed is often limited to a few kHz along a fast axis and a few hundredHertz along a slow axis. In addition, the mechanical-based scanner hasstability and reliability issues. Depth sensing based on a fixedstructured light pattern uses a diffractive optical element to generatea fixed structured light pattern projected into an object space. Thedepth sensing based on the fixed structured light pattern further uses apre-stored look-up table to compute and extract depth maps. However, thedepth sensing based on the fixed structured light pattern and thediffractive optical element is not robust enough for dynamic depthsensing.

SUMMARY

A depth camera assembly (DCA) determines depth information associatedwith one or more objects in a local area. The DCA comprises a lightgenerator, an imaging device and a controller. The light generator isconfigured to illuminate the local area with structured light inaccordance with emission instructions. The light generator comprises anillumination source, an acousto-optic deflector (AOD), a liquid crystaldevice (LCD), and a projection assembly. The illumination source isconfigured to emit one or more optical beams. The AOD generatesdiffracted scanning beams (in one or two dimensions) from the one ormore optical beams emitted from the illumination source. The AOD isconfigured to function as at least one dynamic diffraction grating thatdiffracts the one or more optical beams by at least one diffractionangle to form the diffracted scanning beams based in part on theemission instructions. The LCD includes a plurality of liquid crystalgratings (LCGs). Each LCG in the LCD has an active state in which theLCG is configured to diffract the diffracted scanning beams by anotherdiffraction angle larger than the at least one diffraction angle basedin part on the emission instructions to generate the structured light.The projection assembly is configured to project the structured lightinto the local area. The imaging device is configured to capture one ormore images of portions of the structured light reflected from one ormore objects in the local area. The controller may be coupled to boththe light generator and the imaging device. The controller generates theemission instructions and provides the emission instructions to thelight generator. The controller is also configured to determine depthinformation for the one or more objects based at least in part on thecaptured one or more images.

An eyeglass-type platform representing a near-eye display (NED) canintegrate the DCA. The NED further includes an electronic display and anoptical assembly. The NED may be part of an artificial reality system.The electronic display of the NED is configured to emit image light. Theoptical assembly of the NED is configured to direct the image light toan eye-box of the NED corresponding to a location of a user's eye. Theimage light may comprise the depth information of the one or moreobjects in the local area determined by the DCA.

A head-mounted display (HMD) can further integrate the DCA. The HMDfurther includes an electronic display and an optical assembly. The HMDmay be part of an artificial reality system. The electronic display isconfigured to emit image light. The optical assembly is configured todirect the image light to an eye-box of the HMD corresponding to alocation of a user's eye. The image light may comprise the depthinformation of the one or more objects in the local area determined bythe DCA.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a diagram of a near-eye-display (NED), in accordance with oneor more embodiments.

FIG. 1B is a cross-section of an eyewear of the NED in FIG. 1A, inaccordance with one or more embodiments.

FIG. 2A is a diagram of a head-mounted display (HMD), in accordance withone or more embodiments.

FIG. 2B is a cross section of a front rigid body of the HMD in FIG. 2A,in accordance with one or more embodiments.

FIG. 3A is an example depth camera assembly (DCA), in accordance withone or more embodiments.

FIG. 3B illustrates a scanning field covered by the DCA in FIG. 3A, inaccordance with one or more embodiments.

FIG. 3C illustrates different diffraction settings in the DCA in FIG. 3Ato cover the scanning field in FIG. 3B, in accordance with one or moreembodiments.

FIG. 4 is a flow chart illustrating a process of determining depthinformation of objects in a local area based on ultra-wide field-of-viewscanning, in accordance with one or more embodiments.

FIG. 5 is a block diagram of an artificial reality system in which aconsole operates, in accordance with one or more embodiments.

The figures depict embodiments of the present disclosure for purposes ofillustration only. One skilled in the art will readily recognize fromthe following description that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles, or benefits touted, of the disclosure described herein.

DETAILED DESCRIPTION

Embodiments of the present disclosure may include or be implemented inconjunction with an artificial reality system. Artificial reality is aform of reality that has been adjusted in some manner beforepresentation to a user, which may include, e.g., a virtual reality (VR),an augmented reality (AR), a mixed reality (MR), a hybrid reality, orsome combination and/or derivatives thereof. Artificial reality contentmay include completely generated content or generated content combinedwith captured (e.g., real-world) content. The artificial reality contentmay include video, audio, haptic feedback, or some combination thereof,and any of which may be presented in a single channel or in multiplechannels (such as stereo video that produces a three-dimensional effectto the viewer). Additionally, in some embodiments, artificial realitymay also be associated with applications, products, accessories,services, or some combination thereof, that are used to, e.g., createcontent in an artificial reality and/or are otherwise used in (e.g.,perform activities in) an artificial reality. The artificial realitysystem that provides the artificial reality content may be implementedon various platforms, including a head-mounted display (HMD) connectedto a host computer system, a standalone HMD, a near-eye display (NED), amobile device or computing system, or any other hardware platformcapable of providing artificial reality content to one or more viewers.

A depth camera assembly (DCA) for determining depth information ofobjects in a local area surrounding some or all of the DCA is presentedherein. The DCA includes a light source, one or more cameras and acontroller. The light source includes a laser source and anacousto-optic deflector (AOD) that generates structured light usinglight emitted from the laser source. The AOD can be composed of one ormore acousto-optic devices or plates. Each acousto-optic plate can beconfigured to diffract incident light by a specific diffraction anglecontrolled by, e.g., an electric field applied to the acousto-opticplate. The light source also includes a plurality of active liquidcrystal gratings (LCGs). Adjustments to settings of the plurality ofLCGs determine where the structured light is projected into the localarea. The one or more cameras capture one or more images of portions ofthe structured light reflected from the objects in the local area. Notethat the portions of the structured light can be also scattered from oneor more objects in the local area, wherein scattering represents a formof diffuse reflection. The controller determines depth information basedon the captured one or more images.

In some embodiments, the DCA is integrated into a NED that captures datadescribing depth information in a local area surrounding some or all ofthe NED. The NED further includes an electronic display and an opticalassembly. The NED may be part of an artificial reality system, e.g., anAR system and/or VR system. The electronic display of the NED isconfigured to emit image light. The optical assembly of the NED isconfigured to direct the image light to an eye-box of the NEDcorresponding to a location of a user's eye, the image light comprisingthe depth information of the objects in the local area determined by theDCA.

In some embodiments, the DCA is integrated into a HMD that captures datadescribing depth information in a local area surrounding some or all ofthe HMD. The HMD may be part of an artificial reality system. The HMDfurther includes an electronic display and an optical assembly. Theelectronic display is configured to emit image light. The opticalassembly is configured to direct the image light to an eye-box of theHMD corresponding to a location of a user's eye, the image lightcomprising the depth information of the objects in the local areadetermined by the DCA.

FIG. 1A is a diagram of a NED 100, in accordance with one or moreembodiments. The NED 100 presents media to a user. Examples of mediapresented by the NED 100 include one or more images, video, audio, orsome combination thereof. In some embodiments, audio is presented via anexternal device (e.g., speakers and/or headphones) that receives audioinformation from the NED 100, a console (not shown), or both, andpresents audio data based on the audio information. The NED 100 may bepart of an artificial reality system (not shown). The NED 100 isgenerally configured to operate as an artificial reality NED. In someembodiments, the NED 100 may augment views of a physical, real-worldenvironment with computer-generated elements (e.g., images, video,sound, etc.).

The NED 100 shown in FIG. 1A includes a frame 105 and a display 110. Theframe 105 includes one or more optical elements which together displaymedia to users. The display 110 is configured for users to see thecontent presented by the NED 100. The display 110 generates an imagelight to present media to an eye of the user. The NED 100 also includesa DCA (not shown in FIG. 1A) configured to determine depth informationof a local area surrounding some or all of the NED 100. The NED 100 alsoincludes an illumination aperture 113, and an illumination source of theDCA emits light (e.g., structured light) through the illuminationaperture 113. An imaging device of the DCA captures light from theillumination source that is reflected from the local area, e.g., throughthe imaging aperture 115. Light emitted from the illumination source ofthe DCA through the illumination aperture 113 comprises structuredlight, as discussed in more detail in conjunction with FIG. 3A. Lightreflected from the local area through the imaging aperture 115 andcaptured by the imaging device of the DCA comprises portions of thereflected structured light. The NED 100 may also include an orientationdetection device 120 that generates one or more measurement signals inresponse to motion of the NED 100 and generates information aboutorientation of the NED 100. Examples of the orientation detection device120 include: one or more accelerometers, one or more gyroscopes, one ormore magnetometers, another suitable type of sensor that detects motion,or some combination thereof.

FIG. 1B is a cross section 125 of an eyewear of the NED 100 illustratedin FIG. 1A, in accordance with one or more embodiments. The crosssection 125 includes at least one display assembly 130 integrated intothe display 110, an eye-box 140, and a DCA 150. The eye-box 140 is alocation where an eye 145 is positioned when a user wears the NED 100.In some embodiments, the frame 105 may represent a frame of eye-wearglasses. For purposes of illustration, FIG. 1B shows the cross section125 associated with a single eye 145 and a single display assembly 130,but in alternative embodiments not shown, another display assembly whichis separate from the display assembly 130 shown in FIG. 1B, providesimage light to another eye 145 of the user.

The display assembly 130 is configured to direct the image light to theeye 145 through the eye-box 140. In some embodiments, when the NED 100is configured as an AR NED, the display assembly 130 also directs lightfrom a local area surrounding the NED 100 to the eye 145 through theeye-box 140. The display assembly 130 may be configured to emit imagelight at a particular focal distance in accordance with varifocalinstructions, e.g., provided from a varifocal module (not shown in FIG.1B).

The display assembly 130 may be composed of one or more materials (e.g.,plastic, glass, etc.) with one or more refractive indices thateffectively minimize the weight and present to the user a field of viewof the NED 100. In alternate configurations, the NED 100 includes anoptical assembly with one or more optical elements between the displayassembly 130 and the eye 145. The optical elements may act to, e.g.,correct aberrations in image light emitted from the display assembly130, magnify image light, perform some other optical adjustment of imagelight emitted from the display assembly 130, or some combinationthereof. The example for optical elements may include an aperture, aFresnel lens, a convex lens, a concave lens, a liquid crystal lens, adiffractive element, a waveguide, a filter, a polarizer, a diffuser, afiber taper, one or more reflective surfaces, a polarizing reflectivesurface, a birefringent element, or any other suitable optical elementthat affects image light emitted from the display assembly 130.

The frame 105 further includes a DCA 150 configured to determine depthinformation of one or more objects in a local area surrounding some orall of the NED 100. The DCA 150 includes an illumination source 155, animaging device 160, and a controller 165 that may be coupled to at leastone of the illumination source 155 and the imaging device 160. In someembodiments (now shown in FIG. 1B), the illumination source 155 and theimaging device 160 each may include its own internal controller. In someembodiments (not shown in FIG. 1B), the illumination source 155 and theimaging device 160 can be widely separated, e.g., the illuminationsource 155 and the imaging device 160 can be located in differentassemblies.

The illumination source 155 may be configured to illuminate the localarea with structured light through the illumination aperture 113 inaccordance with emission instructions generated by the controller 165.The illumination source 155 may include a plurality of emitters thateach emits light having certain characteristics (e.g., wavelength,polarization, coherence, temporal behavior, etc.). The characteristicsmay be the same or different between emitters, and the emitters can beoperated simultaneously or individually. In one embodiment, theplurality of emitters could be, e.g., laser diodes (e.g., edgeemitters), inorganic or organic LEDs, a vertical-cavity surface-emittinglaser (VCSEL), or some other source.

The imaging device 160 includes one or more cameras configured tocapture, through the imaging aperture 115, one or more images of atleast a portion of the structured light reflected from one or moreobjects in the local area. In one embodiment, the imaging device 160 isan infrared camera configured to capture images in the infraredspectrum. Additionally or alternatively, the imaging device 160 may bealso configured to capture images of visible spectrum light. The imagingdevice 160 may include a charge-coupled device (CCD) detector, acomplementary metal-oxide-semiconductor (CMOS) detector or some othertypes of detectors (not shown in FIG. 1B). The imaging device 160 may beconfigured to operate with a pre-determined frame rate for fastdetection of objects in the local area.

The controller 165 may generate the emission instructions and providethe emission instructions to the illumination source 155 for controllingoperation of the illumination source 155. The controller 165 maycontrol, based on the emission instructions, operation of theillumination source 155 to dynamically adjust a pattern of thestructured light illuminating the local area, an intensity of the lightpattern, a density of the light pattern, location of the light beingprojected at the local area, etc. The controller 165 may be alsoconfigured to determine depth information for the one or more objects inthe local area based in part on the one or more images captured by theimaging device 160. In some embodiments, the controller 165 provides thedetermined depth information to a console (not shown in FIG. 1B) and/oran appropriate module of the NED 100 (e.g., a varifocal module, notshown in FIG. 1B). The console and/or the NED 100 may utilize the depthinformation to, e.g., generate content for presentation on the display110. More details about the structure and operation of the DCA 150 aredisclosed in conjunction with FIGS. 3A-3C and FIG. 4.

FIG. 2A is a diagram of a HMD 200, in accordance with one or moreembodiments. The HMD 200 may be part of an artificial reality system. Inembodiments that describe AR system and/or a MR system, portions of afront side 202 of the HMD 200 are at least partially transparent in thevisible band (˜380 nm to 750 nm), and portions of the HMD 200 that arebetween the front side 202 of the HMD 200 and an eye of the user are atleast partially transparent (e.g., a partially transparent electronicdisplay). The HMD 200 includes a front rigid body 205, a band 210, and areference point 215. The HMD 200 also includes a DCA configured todetermine depth information of a local area surrounding some or all ofthe HMD 200. The HMD 200 also includes an imaging aperture 220 and anillumination aperture 225, and an illumination source of the DCA emitslight (e.g., structured light) through the illumination aperture 225. Animaging device of the DCA captures light from the illumination sourcethat is reflected from the local area through the imaging aperture 220.Light emitted from the illumination source of the DCA through theillumination aperture 225 comprises structured light, as discussed inmore detail in conjunction with FIG. 3A and FIG. 4. Light reflected fromthe local area through the imaging aperture 220 and captured by theimaging device of the DCA comprises portions of the reflected structuredlight.

The front rigid body 205 includes one or more electronic displayelements (not shown in FIG. 2A), one or more integrated eye trackingsystems (not shown in FIG. 2A), an Inertial Measurement Unit (IMU) 230,one or more position sensors 235, and the reference point 215. In theembodiment shown by FIG. 2A, the position sensors 235 are located withinthe IMU 230, and neither the IMU 230 nor the position sensors 235 arevisible to a user of the HMD 200. The IMU 230 is an electronic devicethat generates fast calibration data based on measurement signalsreceived from one or more of the position sensors 235. A position sensor235 generates one or more measurement signals in response to motion ofthe HMD 200. Examples of position sensors 235 include: one or moreaccelerometers, one or more gyroscopes, one or more magnetometers,another suitable type of sensor that detects motion, a type of sensorused for error correction of the IMU 230, or some combination thereof.The position sensors 235 may be located external to the IMU 230,internal to the IMU 230, or some combination thereof.

FIG. 2B is a cross section 240 of the front rigid body 205 of the HMD200 shown in FIG. 2A. As shown in FIG. 2B, the front rigid body 205includes an electronic display 245 and an optical assembly 250 thattogether provide image light to an eye-box 255. The eye-box 255 is thelocation of the front rigid body 205 where a user's eye 260 ispositioned. For purposes of illustration, FIG. 2B shows a cross section240 associated with a single eye 260, but another optical assembly 250,separate from the optical assembly 250, provides altered image light toanother eye of the user. The front rigid body 205 also has an opticalaxis corresponding to a path along which image light propagates throughthe front rigid body 205.

The electronic display 245 generates image light. In some embodiments,the electronic display 245 includes an optical element that adjusts thefocus of the generated image light. The electronic display 245 displaysimages to the user in accordance with data received from a console (notshown in FIG. 2B). In various embodiments, the electronic display 245may comprise a single electronic display or multiple electronic displays(e.g., a display for each eye of a user). Examples of the electronicdisplay 245 include: a liquid crystal display, an organic light emittingdiode (OLED) display, an inorganic light emitting diode (ILED) display,an active-matrix organic light-emitting diode (AMOLED) display, atransparent organic light emitting diode (TOLED) display, some otherdisplay, a projector, or some combination thereof. The electronicdisplay 245 may also include an aperture, a Fresnel lens, a convex lens,a concave lens, a diffractive element, a waveguide, a filter, apolarizer, a diffuser, a fiber taper, a reflective surface, a polarizingreflective surface, or any other suitable optical element that affectsthe image light emitted from the electronic display. In someembodiments, one or more of the display block optical elements may haveone or more coatings, such as anti-reflective coatings.

The optical assembly 250 magnifies received light from the electronicdisplay 245, corrects optical aberrations associated with the imagelight, and the corrected image light is presented to a user of the HMD200. At least one optical element of the optical assembly 250 may be anaperture, a Fresnel lens, a refractive lens, a reflective surface, adiffractive element, a waveguide, a filter, or any other suitableoptical element that affects the image light emitted from the electronicdisplay 245. Moreover, the optical assembly 250 may include combinationsof different optical elements. In some embodiments, one or more of theoptical elements in the optical assembly 250 may have one or morecoatings, such as anti-reflective coatings, dichroic coatings, etc.Magnification of the image light by the optical assembly 250 allowselements of the electronic display 245 to be physically smaller, weighless, and consume less power than larger displays. Additionally,magnification may increase a field-of-view of the displayed media. Forexample, the field-of-view of the displayed media is such that thedisplayed media is presented using almost all (e.g., 110 degreesdiagonal), and in some cases all, of the user's field-of-view. In someembodiments, the optical assembly 250 is designed so its effective focallength is larger than the spacing to the electronic display 245, whichmagnifies the image light projected by the electronic display 245.Additionally, in some embodiments, the amount of magnification may beadjusted by adding or removing optical elements.

As shown in FIG. 2B, the front rigid body 205 further includes a DCA 265for determining depth information of one or more objects in a local area270 surrounding some or all of the HMD 200. The DCA 265 includes a lightgenerator 275, an imaging device 280, and a controller 285 that may becoupled to both the light generator 275 and the imaging device 280. Thelight generator 275 emits light through the illumination aperture 225.In accordance with embodiments of the present disclosure, the lightgenerator 275 is configured to illuminate the local area 270 withstructured light 290 in accordance with emission instructions generatedby the controller 285. The controller 285 is configured to controloperation of certain components of the light generator 275, based on theemission instructions. The controller 285 provides the emissioninstructions to a plurality of diffractive optical elements of the lightgenerator 275 to control a field-of-view of the local area 270illuminated by the structured light 290. More details about controllingthe plurality of diffractive optical elements of the light generator 275by the controller 285 are disclosed in conjunction with FIGS. 3A-3C andFIG. 4.

The light generator 275 may include a plurality of emitters that eachemits light having certain characteristics (e.g., wavelength,polarization, coherence, temporal behavior, etc.). The characteristicsmay be the same or different between emitters, and the emitters can beoperated simultaneously or individually. In one embodiment, theplurality of emitters could be, e.g., laser diodes (e.g., edgeemitters), inorganic or organic LEDs, a vertical-cavity surface-emittinglaser (VCSEL), or some other source. In some embodiments, a singleemitter or a plurality of emitters in the light generator 275 can emitlight having a structured light pattern. More details about the DCA 265that includes the light generator 275 are disclosed in conjunction withFIG. 3A.

The imaging device 280 includes one or more cameras configured tocapture, through the imaging aperture 220, portions of the structuredlight 290 reflected from the local area 270. The imaging device 280captures one or more images of one or more objects in the local area 270illuminated with the structured light 290. The controller 285 is alsoconfigured to determine depth information for the one or more objectsbased on the captured portions of the reflected structured light. Insome embodiments, the controller 285 provides the determined depthinformation to a console (not shown in FIG. 2B) and/or an appropriatemodule of the HMD 200 (e.g., a varifocal module, not shown in FIG. 2B).The console and/or the HMD 200 may utilize the depth information to,e.g., generate content for presentation on the electronic display 245.

In some embodiments, the front rigid body 205 further comprises an eyetracking system (not shown in FIG. 2B) that determines eye trackinginformation for the user's eye 260. The determined eye trackinginformation may comprise information about an orientation of the user'seye 260 in the eye-box 255, i.e., information about an angle of aneye-gaze. The eye-box 255 represents a three-dimensional volume at anoutput of a HMD in which the user's eye is located to receive imagelight. In one embodiment, the user's eye 260 is illuminated with astructured light. Then, the eye tracking system can use locations of thereflected structured light in a captured image to determine eye positionand eye-gaze. In another embodiment, the eye tracking system determineseye position and eye-gaze based on magnitudes of image light capturedover a plurality of time instants.

In some embodiments, the front rigid body 205 further comprises avarifocal module (not shown in FIG. 2B). The varifocal module may adjustfocus of one or more images displayed on the electronic display 245,based on the eye tracking information. In one embodiment, the varifocalmodule adjusts focus of the displayed images and mitigatesvergence-accommodation conflict by adjusting a focal distance of theoptical assembly 250 based on the determined eye tracking information.In another embodiment, the varifocal module adjusts focus of thedisplayed images by performing foveated rendering of the one or moreimages based on the determined eye tracking information. In yet anotherembodiment, the varifocal module utilizes the depth information from thecontroller 285 to generate content for presentation on the electronicdisplay 245.

FIG. 3A is an example DCA 300 configured for depth sensing based onstructured light with an ultra-wide field-of-view, in accordance withone or more embodiments. The DCA 300 includes a light generator 305, animaging device 310, and a controller 315 coupled to both the lightgenerator 305 and the imaging device 310. The DCA 300 may be configuredto be a component of the NED 100 in FIG. 1A and/or a component of theHMD 200 in FIG. 2A. Thus, the DCA 300 may be an embodiment of the DCA150 in FIG. 1B and/or an embodiment of the DCA 265 in FIG. 2B; the lightgenerator 305 may be an embodiment of the illumination source 155 inFIG. 1B and/or an embodiment of the light generator 275 in FIG. 2B; andthe imaging device 310 may be an embodiment of the imaging device 160 inFIG. 1B and/or an embodiment of the imaging device 280 in FIG. 2B.

The light generator 305 is configured to illuminate and scan a localarea 320 with structured light in accordance with emission instructionsfrom the controller 315. The light generator 305 includes anillumination source 325 (e.g., laser diode) configured to emit one ormore optical beams 330. The illumination source 325 may directlygenerate the one or more optical beams 330 as polarized light. The oneor more optical beams 330 can be circularly polarized (right handed orin other embodiments left handed). In alternate embodiments, the one ormore optical beams 330 can be linearly polarized (vertical andhorizontal), or elliptically polarized (right or left). Alternatively,the illumination source 325 may emit unpolarized light, and a polarizingelement (not shown in FIG. 3A) separate from the illumination source 325may generate the one or more optical beams 330 as polarized light, basedin part on the emission instructions from the controller 315. Thepolarizing element may be integrated into the illumination source 325 orplaced in front of the illumination source 325. In some embodiments, fordepth sensing based on time-of-flight, the one or more optical beams 330are temporally modulated for generating temporally modulatedillumination of the local area 320.

A beam conditioning assembly 335 collects light emitted from theillumination source 325 and directs the collected light toward a portionof an AOD 340. The beam conditioning assembly 335 may be composed of oneor more optical elements, e.g., lenses having specific optical powers.

The AOD 340 diffracts light into one or more dimensions. The AOD 340 iscomposed of one or more acousto-optic devices or plates that generatediffracted scanning beams 345 in one or two dimensions by diffractingthe one or more optical beams 330. In some embodiments, the diffractedscanning beams 345 represent structured light of a defined pattern,e.g., a pattern of light having parallel stripes, a dot pattern, etc. Insome embodiments, the AOD 340 is configured to function as at least onedynamic diffraction grating that diffracts the one or more optical beams330 to form the diffracted scanning beams 345 based in part on emissioninstructions from the controller 315. Each acousto-optic device in theAOD 340 may include a transducer or an array of transducers and one ormore diffraction areas (not shown in FIG. 3A). Responsive to at leastone radio frequency in the emission instructions, the transducer or thearray of transducers of the acousto-optic device in the AOD 340 may beconfigured to generate at least one sound wave in the one or morediffraction areas of the acousto-optic device to form the at least onedynamic diffraction grating.

The AOD 340 can be configured to actively scan a plurality ofdiffraction angles at which the one or more optical beams 330 arediffracted and interfered to form the diffracted scanning beams 345. TheAOD 340 is configured to scan the plurality of diffraction anglesbetween, e.g., −5 degrees and +5 degrees. In this way, the diffractedscanning beams 345 formed by the AOD 340 covers a scanning zone with afield-of-view of, e.g., 10 degrees, along one or two dimensions. In someembodiments, the AOD 340 is configured to scan the plurality ofdiffraction angles with the scanning resolution of 0.1 degree, thussupporting a fine-grained scanning. Due to a relatively narrow scanningzone, the AOD 340 can support fast scanning with scanning speeds, e.g.,in the order of MHz.

The AOD 340 can be used to scan the local area 320 at discrete angles ora continuum of angles, depending on a radio frequency signal that drivesthe AOD 340, controlled by, e.g., the controller 315. In someembodiments, to achieve scanning of discrete angles (e.g., 0, +x, and −xdegrees along x dimension, and 0, +y, −y degrees along y dimension),each acousto-optic device in the AOD 340 is driven by a specific radiofrequency signal having a frequency of, e.g., f_(c), f_(c)+f_(m),f_(c)−f_(m), for scanning of discrete angles along a correspondingdimension. In one or more embodiments, an angle of light incident toeach acousto-optic device in the AOD 340 satisfies a Bragg matchingcondition. Note that f_(c) is a frequency of a carrier signal and f_(m)is a frequency of a modulation signal that modulates the carrier signalfor generating the radio frequency signal that drives an acousto-opticdevice within the AOD 340. The frequency of carrier signal, f_(c), canbe set to be around a center of a frequency bandwidth of the AOD 340 inorder to maximize the diffraction efficiency of the diffracted scanningbeams 345. In addition, the value of 2f_(m) should be smaller than anacoustic resonant 3 dB frequency bandwidth of each acousto-optic devicewithin the AOD 340. Note also that the frequency bandwidth of the AOD340 is relatively narrow, providing a relatively narrow angular spread(band) in the Bragg regime for the diffracted scanning beams 345. Inalternate embodiments, to achieve fast scanning of a continuum of angles(e.g., with resolution smaller than 0.1 degrees, along both x and ydimensions), each acousto-optic device of the AOD 340 is driven by afrequency sweep signal (e.g., controlled by the controller 315) thatperforms frequency sweep during a short time duration. For example, thefrequency sweep signal may cover frequencies from f_(c)−f_(m) tof_(c)+f_(m) with a defined frequency resolution of Δ f_(m) that isrelated to a time bandwidth product of an acousto-optic device of theAOD 340.

In some embodiments, a radio frequency driving power of the AOD 340controlled by the controller 315 can be up to 500 mW, and a drivingradio frequency controlled by the controller 315 can be in the range ofa few MHz up to GHz. In one embodiment, the AOD 340 is configured as adiffraction grating device having an array of transducers. In analternate embodiment, the AOD 340 is configured as a diffraction gratingdevice having a single transducer. The AOD 340 represents a dynamicphase grating suitable for achieving both dynamic and high speedscanning, based on a sound wave traveling through a crystal thatdiffracts the one or more optical beams 330 and creates the diffractedscanning beams 345 as real-time configurable structured light. In someembodiments, the AOD 340 can accept the one or more optical beams 330having visible to infrared wavelengths. In some embodiments, efficiencyof the AOD 340 depends on a bandwidth of each transducer in the AOD 340,which can be designed to maintain efficiency between, e.g., 80% and 90%.

In some embodiments, the one or more optical beams 330 are incident onthe AOD 340 at an angle that satisfies the Bragg matching condition. TheAOD 340 may directly generate the diffracted scanning beams 345 aspolarized light (e.g., circularly polarized light) by orienting the oneor more optical beams 330 to a crystal in the AOD 340 in a geometrysatisfying the Bragg matching condition. Note that the diffractedscanning beams 345 can be either right handed circularly polarized orleft handed circularly polarized based on the crystal in the AOD 340. Insome embodiments, a state of polarization (SOP) of the one or moreoptical beams 330 incident to the AOD 340 matches an eigenstate ofpolarization at the Bragg angle for achieving maximum diffractionefficiency of the AOD 340. A polarization element can be included infront of the AOD 340 (not shown in FIG. 3A) to maximize the diffractionefficiency of the AOD 340, if the SOP of the one or more optical beams330 does not match the eigenstate of polarization at the Bragg angle forphase matching condition.

The AOD 340 provides ultrafast scanning speed to dynamically scanarbitrary patterns that can be projected to one or more objects in thelocal area 320. The AOD 340 operates based on one or more differentcrystal types, each crystal type having a wide spectral bandwidth andbeing transparent for light from visible to infrared wavelengths. Hence,the AOD 340 can diffract the one or more optical beams 330 in a widewavelength range. Depending on a configuration of the NED 100 and/or theHMD 200, the AOD 340 can be implemented as a different type of device.In one embodiment, the AOD 340 is implemented as a bulk device. Inanother embodiment, the AOD 340 is implemented as a plate device, i.e.,a compact version of a bulk device. In yet another embodiment, the AOD340 is implemented as a thin film device based on a surface propagatingacoustic wave deflector. Depending on a type of scanning, the AOD 340may include a different number of acousto-optic devices or elements. Inone embodiment, the AOD 340 includes a single acousto-optic device orelement for generating the diffracted scanning beams 345 asone-dimensional scanning beams for one-dimensional random scanning. Inan alternate embodiment, the AOD 340 includes at least one pair ofacousto-optic devices whose axes of orientation are orthogonal to eachother. Accordingly, one acousto-optic device in a pair of acousto-opticdevices diffracts light in one dimension (e.g., x) and the secondacousto-optic device in the pair diffracts the x-diffracted light alongan orthogonal dimension (e.g., y), thereby generating the diffractedscanning beams 345 as two-dimensional scanning beams for two-dimensionalrandom scanning. Each acousto-optic device or element in the AOD 340 canbe configured to function as a dynamic diffraction grating thatdiffracts incident light by a specific diffraction angle based in parton the emission instructions from the controller 315. Additional detailsregarding structure and operation of an acousto-optic device isdescribed with regard to U.S. application Ser. No. 15/599,353, filed onMay 18, 2017, and U.S. application Ser. No. 15/643,912, filed on Jul. 7,2017, which are incorporated by reference in their entireties.

A liquid crystal device (LCD) 350 is positioned in front of the AOD 340.The LCD 350 is configured to further diffract light received from theAOD 340, based in part on the emission instructions from the controller315. The LCD 350 diffracts the diffracted scanning beams 345 to generatestructured light 355 having an ultra-wide field-of-view for scanning thelocal area 320. It should be understood that the structured light 355with the ultra-wide field-of-view can be generated by the DCA 300 havinga reverse order of the AOD 340 and the LCD 350 where the AOD 340 ispositioned in front of the LCD 350 (the embodiment not shown in FIG.3A).

The LCD 350 includes a plurality of active liquid crystal gratings(LCGs) in an optical series. Note that an active LCG is in opticalseries with another active LCG when light diffracted by the active LCGis incident to the other active LCG. Each active LCG in the LCD 350 isconfigured to further diffract the diffracted scanning beams 345 by aspecific diffraction angle, which can be controlled based in part on theemission instructions from the controller 315. In some embodiments, eachactive LCG can be made based on, e.g., photo-alignment with liquidcrystal polymers and a polarization holography setup. In someembodiments, the LCD 350 includes three active LCGs, thus forming withthe AOD 340 a series of four active diffraction layers. At any timeinstant, two diffraction layers out of four diffraction layers may be inan active state (e.g., controlled by the controller 315) and generatethe structured light 355 covering an ultra-wide scanning field of thelocal area 320. In some embodiments, at most two of LCGs in the LCD 350are in active states providing diffraction of corresponding incidentlight.

In some embodiments, based on modulation of optical axis, the LCGs inthe LCD 350 can be Pancharatnam-Berry phase gratings, polarizationvolume gratings, or conventional LCGs. In some other embodiments, basedon refractive index modulation, the LCGs in the LCD 350 can beconventional LCGs with patterned indium tin oxide (ITO) films or hiddendielectric pattern to generate in-homogenous electric field across asubstrate for producing grating effect. In some other embodiments, basedon modulation of thickness, the LCGs in the LCD 350 can be implementedusing liquid crystal cells with in-homogenous cell gap across asubstrate for generating grating effect. In some other embodiments, theLCGs in the LCD 350 can be implemented by filling liquid crystals intosubstrates having a grating structure on one or both sides of thesubstrates. In some other embodiments, the LCGs in the LCD 350 can bepolarization sensitive or polarization non-sensitive depending on aconfiguration and polarization state of the illumination source 325.

The LCD 350 may utilize switchable or non-switchable liquid crystalcells, depending on the working principle of LCGs within the LCD 350. Inone or more embodiments, the switchable liquid crystal cells function asa switchable half-wave plate. In some embodiments, the light generator305 of the DCA 300 may include polarization correction elements,depending on the working principle of LCGs in the LCD 350, layer stackconfiguration of the AOD 340 and/or the LCD 350, and a polarizationstate of the illumination source 325. A polarization correction elementintegrated into the DCA 300 may be a linear polarizer, a circularpolarizer, a quarter waveplate, a c-plate, or combination thereof.

A projection assembly 360 is positioned in front of the combination ofthe AOD 340 and the LCD 350. The projection assembly 360 includes one ormore optical elements (lenses). Optionally, the projection assembly 360includes a polarizing element for polarization of the structured light355. The structured light 355 may be selected from a group consisting oflinearly polarized light (vertical and horizontal), right handedcircularly polarized light, left handed circularly polarized light, andelliptically polarized light. The projection assembly 360 projects thestructured light 355 into the local area 320 over an ultra-widefield-of-view, e.g., of 160 degrees along x dimension and/or ydimension. The structured light 355 illuminates portions of the localarea 320, including one or more objects in the local area 320. Areflected structured light 365 is generated based on reflection of thestructured light 355 from the one or more objects in the local area 320.

The imaging device 310 captures one or more images of the one or moreobjects in the local area 320 by capturing the portions of the reflectedstructured light 365. In one embodiment, the imaging device 310 is aninfrared camera configured to capture images in an infrared spectrum. Inanother embodiment, the imaging device 310 is configured to capture animage light of a visible spectrum. The imaging device 310 can beconfigured to operate with a frame rate in the range of kHz to MHz forfast detection of objects in the local area 320. In an embodiment, theimaging device 310 includes a polarizing element placed in front of acamera for receiving and propagating the reflected structured light 365of a particular polarization. The reflected structured light 365 may beselected from a group consisting of linearly polarized light (verticaland horizontal), right handed circularly polarized light, left handedcircularly polarized light, and elliptically polarized light. It shouldbe noted that polarization of the reflected structured light 365 can bedifferent than polarization of the structured light 355 that illuminatesthe local area 320. In some embodiments, the imaging device 310 includesmore than one camera.

In some embodiments, the DCA 300 includes a light shutter (not shown inFIG. 3A) coupled to the imaging device 310. The light shutter canoperate such that the NED 100 or the HMD 200 switches between an AR modeand a VR mode. In one embodiment, the light shutter is implemented as amechanical component. In the AR mode, the mechanical light shutter maybe open and portions of the reflected structured light 365 propagates toa detector of the imaging device 310. In the VR mode, the mechanicallight shutter may be closed to block one or more portions of thereflected structured light 365 from reaching the detector of the imagingdevice 310. In another embodiment, the light shutter is implemented as apolarizer configured to propagate light of specific polarization, e.g.,based on polarization instructions from the controller 315. In the ARmode, the light shutter implemented as a polarizer may be configured topropagate light having the same polarization as one or more portions ofthe reflected structured light 365. In the VR mode, the light shutterimplemented as a polarizer may be configured to block propagation oflight having the same polarization as one or more portions of thereflected structured light 365.

The controller 315 is configured to control operations of variouscomponents of the DCA 300 in FIG. 3A. In some embodiments, thecontroller 315 provides emission instructions to the illumination source325 to control intensity of the one or more optical beams 330,modulation of the one or more optical beams 330, a time duration duringwhich the illumination source 325 is activated, etc. The controller 315may further create the emission instructions which include a radiofrequency at which the AOD 340 is driven. The controller 315 maygenerate the emission instructions based on, e.g., a predetermined listof values for the radio frequency stored in a look-up table of thecontroller 315. In an embodiment, the predetermined radio frequenciesare stored as waveforms in an electronic chip, e.g., in a direct digitalsynthesizer (not shown in FIG. 3A) coupled to the controller 315. Inanother embodiment, the emission instructions are created by a voicecontrol integrated into the controller 315. Upon a verbal request, thevoice control of the controller 315 computes a radio frequency fordriving the AOD 340 to generate the diffracted scanning beams 345 andthe structured light 355 of a specific spatial frequency suitable fordetection of stationary object(s) and/or tracking of moving object(s) inthe local area 320 at a certain distance from the imaging device 310.

The controller 315 can modify the radio frequency at which the AOD 340is driven to adjust a diffraction angle at which the one or more opticalbeams 330 are diffracted. In this way, the controller 315 can instructthe AOD 340 to scan a plurality of diffraction angles at which the oneor more optical beams 330 are diffracted and interfered to form thediffracted scanning beams 345 and the structured light 355. A radiofrequency at which the AOD 340 is driven may control a separation of theoptical beams 330 diffracted by the AOD 340. Hence, a spatial frequencyof the resulting diffracted scanning beams 345 (and of the structuredlight 355) directly depends on the radio frequency at which the AOD 340is driven.

As shown in FIG. 3A, the controller 315 is further coupled to theimaging device 310 and can be configured to determine depth informationfor the one or more objects in the local area 320. The controller 315 isconfigured to determine depth information for the one or more objectsbased at least in part on the captured one or more images of portions ofthe reflected structured light 365. In some embodiments, the controller315 can be configured to determine the depth information based onpolarization information of the reflected structured light 365 andpolarization information of the structured light 355. For a depthsensing method based on structured light illumination, the controller315 is configured to determine the depth information based onphase-shifted patterns of the portions of the reflected structured light365 distorted by shapes of the one or more objects in the local area320, and to use triangulation calculation to obtain a depth map of thelocal area 320.

FIG. 3B illustrates a scanning field 370, which may be covered by thestructured light 355 generated by the DCA 300 in FIG. 3A, in accordancewith one or more embodiments. As shown in FIG. 3B, the scanning field370 includes a plurality of sub-zones 375. In the illustrativeembodiment shown in FIG. 3B, the scanning field 370 covers afield-of-view of 160 degrees along both x and y axes, whereas eachsub-zone 375 covers 10 degrees along y axis. As discussed above inconjunction with FIG. 3A, the controller 315 configures, via theemission instructions, the AOD 340 to scan a plurality of diffractionangles at which the one or more optical beams 330 are diffracted andinterfered to form the diffracted scanning beams 345. In the embodimentillustrated in FIG. 3B, the AOD 340 scans diffraction angles between −5degrees and +5 degrees, e.g., with scanning resolution of 0.1 degreesand scanning speed in the order of MHz. In this way, the AOD 340 isconfigured to achieve fast and fine-grained scanning within each subzone375 of the scanning field 370.

As also discussed above in conjunction with FIG. 3A, the controller 315further configures, via the emission instructions, each active LCGwithin the LCD 350 to further diffract the diffracted scanning beams 345by a specific diffraction angle at a particular time instant. In someembodiments, diffraction angles achieved by each active LCG within theLCD 350 are larger than one or more diffraction angles achieved by theAOD 340. In this way, the LCD 350 is configured to generate thestructured light 355 that scans the local area 320 from one sub-zone 375to another (not necessarily adjacent) sub-zone 375 of the scanning field370, with a scanning speed, e.g., in the order of kHz. Thus, the LCD 350with the plurality of active LCGs enables scanning with an ultra-widefield-of-view and a large diffraction angle. In some embodiments, a sizeof the AOD 340 and a size of the LCD 350 are of sub-millimeter order.Power consumption of the AOD 340 and the LCD 350 is, e.g., between 10 mWand 100 mW. An active LCG within the LCD 350 having a large diffractionangle (e.g., of 75 degrees) can achieve high efficiency by using highlybirefringent and dual twist or multiple twisted structures.

FIG. 3C illustrates a table 380 with different diffraction settings ofthe AOD 340 and the active LCGs within the LCD 350 for covering thescanning field 370 in FIG. 3B having a field-of-view between −80 degreesand +80 degrees, in accordance with one or more embodiments. Note thatcolumns in the table 380 in FIG. 3C correspond to different timeinstants, e.g., time instants t⁻⁸⁰, t⁻⁷⁰, . . . , t₀, . . . , t₆₀, t₇₀,t₈₀, not necessarily in chronological order. In the illustrativeembodiment of FIG. 3C, the AOD 340 is configured to diffract the one ormore optical beams 330 by a diffraction angle between −5 degrees and +5degrees, with the resolution of, e.g., 0.1 degree to form the diffractedscanning beams 345 (and the structured light 355) for fast andfine-grained scanning within any sub-zone 375 of the scanning field 370in FIG. 3B. In the illustrative embodiment of FIG. 3C, the LCD 350includes three active LCGs, e.g., LCG1, LCG2, LCG3, in optical series;LCG1 provides a fixed diffraction angle of either −15 degrees or +15degrees; LCG2 provides a fixed diffraction angle of either −35 degreesor +35 degrees; and LCG3 provides a fixed diffraction angle of either−75 degrees or +75 degrees. In this way, the combination of the AOD 340and the LCD 350 generates the structured light 355 that scans the localarea 320 from one sub-zone 375 to another (not necessarily adjacent)sub-zone 375 of the scanning field 370, with a fast scanning speedwithin each sub-zone 375 and a moderate speed of switching betweensub-zones 375.

In some embodiments, the controller 315 of the DCA 300 in FIG. 3Aadjusts, based in part on the emission instructions over a plurality oftime instants, settings of the AOD 340 and of each active LCG in the LCD350 to generate the structured light 355 for covering the scanning field370 in FIG. 3B in the plurality of time instants. As shown, e.g., in thefirst column of the table 380 in FIG. 3C corresponding to a time instantt⁻⁸⁰, the controller 315 adjusts the settings of the AOD 340 and of theLCD 350 so that a diffraction angle of the AOD 340 is −5 degrees, LCG1and LCG2 are in an inactive state, and a diffraction angle of LCG3 is−75 degrees. Thus, at the time instant t⁻⁸⁰, a total diffraction anglegenerated by the combination of the AOD 340 and the LCD 350 is −80degrees. Then, the controller 315 adjusts the settings of the AOD 340 tochange, during a plurality of time instants, e.g., with MHz speed andresolution of 0.1 degree, a diffraction angle between −5 degrees and +5degrees. In the same time, the controller 315 does not adjust thesettings of the active LCGs within the LCD 350, i.e., LCG1 and LCG2 arestill in the inactive state and a diffraction angle of LCG3 is still −75degrees. In this way, the structured light 355 generated by thecombination of the AOD 340 and the LCD 350 scans one sub-zone 375 of thescanning field 370 in FIG. 3B.

The scanning process can continue by switching from one sub-zone 375 toanother (not necessarily adjacent) sub-zone 375 in FIG. 3B in accordancewith the settings shown in the table 380 in FIG. 3C until the entirescanning field 370 in FIG. 3B is covered by the structured light 355. Ateach time instant shown in FIG. 3C, a different sub-zone 375 of thescanning field 370 in FIG. 3B is covered. As shown in FIG. 3C, at eachtime instant, at most two diffraction layers out of three active LCGs inthe LCD 350 are in active states. Because the AOD 340 and the LCGs inthe LCD 350 are active components, the AOD 340 and each active LCG inthe LCD 350 can be configured to be in active state or in inactive statebased on an electrical field, controlled by, e.g., the controller 315.

In the illustrative embodiment of FIG. 3C, the AOD 340 is in activestate for scanning small angles and achieving fast and fine-grainedscanning within any sub-zone 375 of the scanning field 370 in FIG. 3B.The AOD 340 may be configured to scan continuously over a range from −5degrees to +5 degrees within the sub-zone 375. Thus, to achieve fast andcontinuous subzone scanning, the AOD 340 is in active state. In someembodiments, the AOD 340 and the LCD 350 are configured to operate in adiscrete scanning mode, where the sub-zone 375 is not scanned by the AOD340 and at least one LCG in the LCD 350 is in active state. In thediscrete scanning mode, the LCD 350 can scan discretely one or morebeams of the structured light 355 in large angles (e.g., from −50degrees to −60 degrees) without scanning within the sub-zone 375. In thediscrete scanning mode, the AOD 340 can be deactivated to save power. Inthe power save mode when the AOD 340 is in inactive state, the zerothorder un-diffracted beam 345 from the AOD 340 is scanned by at least oneactive LCG of the LCD 350.

FIG. 4 is a flow chart illustrating a process 400 of determining depthinformation of objects in a local area based on ultra-wide field-of-viewscanning, which may be implemented at the NED 100 shown in FIG. 1Aand/or the HMD 200 shown in FIG. 2A, in accordance with one or moreembodiments. The process 400 of FIG. 4 may be performed by thecomponents of a DCA (e.g., the DCA 300). Other entities (e.g., a NED, aHMD, and/or console) may perform some or all of the steps of the processin other embodiments. Likewise, embodiments may include different and/oradditional steps, or perform the steps in different orders.

The DCA generates 410 (e.g., via a controller) emission instructions.The DCA may provide the emission instructions to an illumination sourcewithin the DCA. Based on the emission instructions, the illuminationsource may emit one or more optical beams. In some embodiments, the DCAgenerates the emission instructions which include information about atleast one radio frequency.

The DCA provides 420 (e.g., via the controller) the emissioninstructions to an AOD to generate diffracted scanning beams from one ormore optical beams by diffracting the one or more optical beams by atleast one diffraction angle using the AOD to form the diffractedscanning beams based in part on the emission instructions. Responsive tothe at least one radio frequency in the emission instructions, the DCAgenerates at least one sound wave within the AOD to form at least onedynamic diffraction grating. In some embodiments, the DCA modifies theat least one radio frequency in the emission instructions to adjust adiffraction angle of the at least one diffraction angle at which the oneor more optical beams are diffracted in each dimension and interfered toform the diffracted scanning beams.

The DCA provides 430 (e.g., via the controller) the emissioninstructions to a LCD comprising a plurality of LCGs, each LCG having anactive state in which the LCG diffracts the diffracted scanning beams byanother diffraction angle larger than the at least one diffraction anglebased in part on the emission instructions to generate the structuredlight. Each LCG is configured to diffract the diffracted scanning beamsby a fixed diffraction angle larger than the at least one diffractionangle based in part on the emission instructions to generate thestructured light.

The DCA may project (e.g., via a projection assembly) the structuredlight into a local area. In some embodiments, the DCA projects thestructured light to illuminate an ultra-wide field-of-view of the localarea for accurate depth scanning. The DCA may also control (e.g., viathe controller) a scanning sub-zone of the ultra-wide field-of-view ofthe local area by controlling dynamic diffraction gratings of the AODand LCGs within the DCA.

The DCA captures 440 (e.g., via an imaging device) one or more images ofportions of the structured light reflected from one or more objects inthe local area. In some embodiments, the imaging device of the DCAincludes a polarizing element and a camera, wherein the polarizingelement is positioned in front of the camera. The polarizing element isconfigured to receive the portions of the reflected structured lighthaving a specific polarization and to propagate the received portions ofreflected polarized light pattern to the camera.

The DCA determines 450 (e.g., via the controller) depth information forthe one or more objects based at least in part on the captured one ormore images. In some embodiments, the DCA determines 450 the depthinformation for the one or more objects based in part on polarizationinformation of the captured portions of the reflected structured light.

In some embodiments, the DCA is configured as part of a NED, e.g., theNED 100 in FIG. 1A. In some other embodiments, the DCA is configured aspart of a HMD, e.g., the HMD 200 in FIG. 2A. In one or more embodiments,the DCA provides the determined depth information to a console coupledto the NED or the HMD. The console is then configured to generatecontent for presentation on an electronic display of the NED or the HMD,based on the depth information. In another embodiment, the DCA providesthe determined depth information to a module of the NED or the HMD thatgenerates content for presentation on the electronic display of the NEDor the HMD, based on the depth information. In an alternate embodiment,the DCA is integrated into a HMD as part of an artificial realitysystem. In this case, the DCA may be configured to sense and displayobjects behind a head of a user wearing the HMD or display objectsrecorded previously.

System Environment

FIG. 5 is a block diagram of one embodiment of an artificial realitysystem 500 in which a console 510 operates. The artificial realitysystem 500 may be a NED system or a HMD system. The artificial realitysystem 500 may operate in an artificial reality system environment. Insome embodiments, the artificial reality system 500 shown by FIG. 5comprises a HMD 505 and an input/output (I/O) interface 515 that iscoupled to the console 510. In some other embodiments (not shown in FIG.5), the artificial reality system 500 includes a NED coupled to theconsole 510. While FIG. 5 shows an example HMD system 500 including oneHMD 505 and on I/O interface 515, in other embodiments any number ofthese components may be included in the artificial reality system 500.For example, there may be multiple HMDs 505 each having an associatedI/O interface 515, with each HMD 505 and I/O interface 515 communicatingwith the console 510. In alternative configurations, different and/oradditional components may be included in the artificial reality system500. Additionally, functionality described in conjunction with one ormore of the components shown in FIG. 5 may be distributed among thecomponents in a different manner than described in conjunction with FIG.5 in some embodiments. For example, some or all of the functionality ofthe console 510 is provided by the HMD 505.

The HMD 505 is a head-mounted display that presents content to a usercomprising virtual and/or augmented views of a physical, real-worldenvironment with computer-generated elements (e.g., two-dimensional (2D)or three-dimensional (3D) images, 2D or 3D video, sound, etc.). In someembodiments, the presented content includes audio that is presented viaan external device (e.g., speakers and/or headphones) that receivesaudio information from the HMD 505, the console 510, or both, andpresents audio data based on the audio information. The HMD 505 maycomprise one or more rigid bodies, which may be rigidly or non-rigidlycoupled together. A rigid coupling between rigid bodies causes thecoupled rigid bodies to act as a single rigid entity. In contrast, anon-rigid coupling between rigid bodies allows the rigid bodies to moverelative to each other. An embodiment of the HMD 505 is the HMD 200described above in conjunction with FIG. 2A.

The HMD 505 includes a DCA 520, an electronic display 525, an opticalassembly 530, one or more position sensors 535, an IMU 540, an optionaleye tracking system 545, and an optional varifocal module 550. Someembodiments of the HMD 505 have different components than thosedescribed in conjunction with FIG. 5. Additionally, the functionalityprovided by various components described in conjunction with FIG. 5 maybe differently distributed among the components of the HMD 505 in otherembodiments.

The DCA 520 captures data describing depth information of an areasurrounding some or all of the HMD 505. The DCA 520 can compute thedepth information using the data (e.g., based on captured portions ofstructured light), or the DCA 520 can send this information to anotherdevice such as the console 510 that can determine the depth informationusing the data from the DCA 520.

The DCA 520 includes a light generator, an imaging device and acontroller that may be coupled to both the light generator and theimaging device. The light generator of the DCA 520 is configured toilluminate a local area with structured light in accordance withemission instructions from the controller. The light generator comprisesan illumination source, an AOD, an LCD, and a projection assembly. Theillumination source is configured to emit one or more optical beams. TheAOD generates diffracted scanning beams from the one or more beamsemitted from the illumination source. The AOD is configured to functionas a dynamic diffraction grating that diffracts the one or more opticalbeams to form the diffracted scanning beams based in part on theemission instructions. The LCD includes a plurality of active LCGs. EachLCG is configured to diffract the diffracted scanning beams based inpart on the emission instructions to generate the structured light. Theprojection assembly is configured to project the structured light intothe local area. The imaging device of the DCA 520 is configured tocapture one or more images of portions of the structured light reflectedfrom one or more objects in the local area. The controller of the DCA520 generates the emission instructions and provides the emissioninstructions to the light generator of the DCA 520. The controller ofthe DCA 520 is also configured to determine depth information for theone or more objects based at least in part on the captured one or moreimages of portions of the reflected structured light. The DCA 520 may bean embodiment of the DCA 300 in FIG. 3A.

The electronic display 525 displays 2D or 3D images to the user inaccordance with data received from the console 510. In variousembodiments, the electronic display 525 comprises a single electronicdisplay or multiple electronic displays (e.g., a display for each eye ofa user). Examples of the electronic display 525 include: a liquidcrystal display, an OLED display, an ILED display, an AMOLED display, aTOLED display, some other display, or some combination thereof. Theelectronic display 525 may be an embodiment of the electronic display245 in FIG. 2B.

The optical assembly 530 magnifies image light received from theelectronic display 525, corrects optical errors associated with theimage light, and presents the corrected image light to a user of the HMD505. The optical assembly 530 includes a plurality of optical elements.Example optical elements included in the optical assembly 530 include:an aperture, a Fresnel lens, a convex lens, a concave lens, a filter, areflecting surface, or any other suitable optical element that affectsimage light. Moreover, the optical assembly 530 may include combinationsof different optical elements. In some embodiments, one or more of theoptical elements in the optical assembly 530 may have one or morecoatings, such as partially reflective or anti-reflective coatings.

Magnification and focusing of the image light by the optical assembly530 allows the electronic display 525 to be physically smaller, weighless and consume less power than larger displays. Additionally,magnification may increase the field-of-view of the content presented bythe electronic display 525. For example, the field-of-view of thedisplayed content is such that the displayed content is presented usingalmost all (e.g., approximately 110 degrees diagonal), and in some casesall, of the user's field-of-view. Additionally in some embodiments, theamount of magnification may be adjusted by adding or removing opticalelements.

In some embodiments, the optical assembly 530 may be designed to correctone or more types of optical error. Examples of optical error includebarrel or pincushion distortions, longitudinal chromatic aberrations, ortransverse chromatic aberrations. Other types of optical errors mayfurther include spherical aberrations, chromatic aberrations or errorsdue to the lens field curvature, astigmatisms, or any other type ofoptical error. In some embodiments, content provided to the electronicdisplay 525 for display is pre-distorted, and the optical assembly 530corrects the distortion when it receives image light from the electronicdisplay 525 generated based on the content. The optical assembly 530 maybe an embodiment of the optical assembly 250 in FIG. 2B.

The IMU 540 is an electronic device that generates data indicating aposition of the HMD 505 based on measurement signals received from oneor more of the position sensors 535 and from depth information receivedfrom the DCA 520. A position sensor 535 generates one or moremeasurement signals in response to motion of the HMD 505. Examples ofposition sensors 535 include: one or more accelerometers, one or moregyroscopes, one or more magnetometers, another suitable type of sensorthat detects motion, a type of sensor used for error correction of theIMU 540, or some combination thereof. The position sensors 535 may belocated external to the IMU 540, internal to the IMU 540, or somecombination thereof.

Based on the one or more measurement signals from one or more positionsensors 535, the IMU 540 generates data indicating an estimated currentposition of the HMD 505 relative to an initial position of the HMD 505.For example, the position sensors 535 include multiple accelerometers tomeasure translational motion (forward/back, up/down, left/right) andmultiple gyroscopes to measure rotational motion (e.g., pitch, yaw,roll). In some embodiments, the IMU 540 rapidly samples the measurementsignals and calculates the estimated current position of the HMD 505from the sampled data. For example, the IMU 540 integrates themeasurement signals received from the accelerometers over time toestimate a velocity vector and integrates the velocity vector over timeto determine an estimated current position of a reference point on theHMD 505. Alternatively, the IMU 540 provides the sampled measurementsignals to the console 510, which interprets the data to reduce error.The reference point is a point that may be used to describe the positionof the HMD 505. The reference point may generally be defined as a pointin space or a position related to the HMD's 505 orientation andposition.

The IMU 540 receives one or more parameters from the console 510. Theone or more parameters are used to maintain tracking of the HMD 505.Based on a received parameter, the IMU 540 may adjust one or more IMUparameters (e.g., sample rate). In some embodiments, certain parameterscause the IMU 540 to update an initial position of the reference pointso it corresponds to a next position of the reference point. Updatingthe initial position of the reference point as the next calibratedposition of the reference point helps reduce accumulated errorassociated with the current position estimated the IMU 540. Theaccumulated error, also referred to as drift error, causes the estimatedposition of the reference point to “drift” away from the actual positionof the reference point over time. In some embodiments of the HMD 505,the IMU 540 may be a dedicated hardware component. In other embodiments,the IMU 540 may be a software component implemented in one or moreprocessors. The IMU 540 may be an embodiment of the IMU 230 in FIG. 2A.

In some embodiments, the eye tracking system 545 is integrated into theHMD 505. The eye tracking system 545 determines eye tracking informationassociated with an eye of a user wearing the HMD 505. The eye trackinginformation determined by the eye tracking system 545 may compriseinformation about an orientation of the user's eye, i.e., informationabout an angle of an eye-gaze. In some embodiments, the eye trackingsystem 545 is integrated into the optical assembly 530. An embodiment ofthe eye-tracking system 545 may comprise an illumination source and animaging device (camera).

In some embodiments, the varifocal module 550 is further integrated intothe HMD 505. The varifocal module 550 may be coupled to the eye trackingsystem 545 to obtain eye tracking information determined by the eyetracking system 545. The varifocal module 550 may be configured toadjust focus of one or more images displayed on the electronic display525, based on the determined eye tracking information obtained from theeye tracking system 545. In this way, the varifocal module 550 canmitigate vergence-accommodation conflict in relation to image light. Thevarifocal module 550 can be interfaced (e.g., either mechanically orelectrically) with at least one of the electronic display 525 and atleast one optical element of the optical assembly 530. Then, thevarifocal module 550 may be configured to adjust focus of the one ormore images displayed on the electronic display 525 by adjustingposition of at least one of the electronic display 525 and the at leastone optical element of the optical assembly 530, based on the determinedeye tracking information obtained from the eye tracking system 545. Byadjusting the position, the varifocal module 550 varies focus of imagelight output from the electronic display 525 towards the user's eye. Thevarifocal module 550 may be also configured to adjust resolution of theimages displayed on the electronic display 525 by performing foveatedrendering of the displayed images, based at least in part on thedetermined eye tracking information obtained from the eye trackingsystem 545. In this case, the varifocal module 550 provides appropriateimage signals to the electronic display 525. The varifocal module 550provides image signals with a maximum pixel density for the electronicdisplay 525 only in a foveal region of the user's eye-gaze, whileproviding image signals with lower pixel densities in other regions ofthe electronic display 525. In one embodiment, the varifocal module 550may utilize the depth information obtained by the DCA 520 to, e.g.,generate content for presentation on the electronic display 525.

The I/O interface 515 is a device that allows a user to send actionrequests and receive responses from the console 510. An action requestis a request to perform a particular action. For example, an actionrequest may be an instruction to start or end capture of image or videodata or an instruction to perform a particular action within anapplication. The I/O interface 515 may include one or more inputdevices. Example input devices include: a keyboard, a mouse, a gamecontroller, or any other suitable device for receiving action requestsand communicating the action requests to the console 510. An actionrequest received by the I/O interface 515 is communicated to the console510, which performs an action corresponding to the action request. Insome embodiments, the I/O interface 515 includes an IMU 540 thatcaptures calibration data indicating an estimated position of the I/Ointerface 515 relative to an initial position of the I/O interface 515.In some embodiments, the I/O interface 515 may provide haptic feedbackto the user in accordance with instructions received from the console510. For example, haptic feedback is provided when an action request isreceived, or the console 510 communicates instructions to the I/Ointerface 515 causing the I/O interface 515 to generate haptic feedbackwhen the console 510 performs an action.

The console 510 provides content to the HMD 505 for processing inaccordance with information received from one or more of: the DCA 520,the HMD 505, and the I/O interface 515. In the example shown in FIG. 5,the console 510 includes an application store 555, a tracking module560, and an engine 565. Some embodiments of the console 510 havedifferent modules or components than those described in conjunction withFIG. 5. Similarly, the functions further described below may bedistributed among components of the console 510 in a different mannerthan described in conjunction with FIG. 5.

The application store 555 stores one or more applications for executionby the console 510. An application is a group of instructions, that whenexecuted by a processor, generates content for presentation to the user.Content generated by an application may be in response to inputsreceived from the user via movement of the HMD 505 or the I/O interface515. Examples of applications include: gaming applications, conferencingapplications, video playback applications, or other suitableapplications.

The tracking module 560 calibrates the HMD system 500 using one or morecalibration parameters and may adjust one or more calibration parametersto reduce error in determination of the position of the HMD 505 or ofthe I/O interface 515. For example, the tracking module 560 communicatesa calibration parameter to the DCA 520 to adjust the focus of the DCA520 to more accurately determine positions of structured light elementscaptured by the DCA 520. Calibration performed by the tracking module560 also accounts for information received from the IMU 540 in the HMD505 and/or an IMU 540 included in the I/O interface 515. Additionally,if tracking of the HMD 505 is lost (e.g., the DCA 520 loses line ofsight of at least a threshold number of structured light elements), thetracking module 560 may re-calibrate some or all of the HMD system 500.

The tracking module 560 tracks movements of the HMD 505 or of the I/Ointerface 515 using information from the DCA 520, the one or moreposition sensors 535, the IMU 540 or some combination thereof. Forexample, the tracking module 550 determines a position of a referencepoint of the HMD 505 in a mapping of a local area based on informationfrom the HMD 505. The tracking module 560 may also determine positionsof the reference point of the HMD 505 or a reference point of the I/Ointerface 515 using data indicating a position of the HMD 505 from theIMU 540 or using data indicating a position of the I/O interface 515from an IMU 540 included in the I/O interface 515, respectively.Additionally, in some embodiments, the tracking module 560 may useportions of data indicating a position or the HMD 505 from the IMU 540as well as representations of the local area from the DCA 520 to predicta future location of the HMD 505. The tracking module 560 provides theestimated or predicted future position of the HMD 505 or the I/Ointerface 515 to the engine 555.

The engine 565 generates a 3D mapping of the area surrounding some orall of the HMD 505 (i.e., the “local area”) based on informationreceived from the HMD 505. In some embodiments, the engine 565determines depth information for the 3D mapping of the local area basedon information received from the DCA 520 that is relevant for techniquesused in computing depth. The engine 565 may calculate depth informationusing one or more techniques in computing depth from one or morepolarized structured light patterns. In various embodiments, the engine565 uses the depth information to, e.g., update a model of the localarea, and generate content based in part on the updated model.

The engine 565 also executes applications within the HMD system 500 andreceives position information, acceleration information, velocityinformation, predicted future positions, or some combination thereof, ofthe HMD 505 from the tracking module 560. Based on the receivedinformation, the engine 565 determines content to provide to the HMD 505for presentation to the user. For example, if the received informationindicates that the user has looked to the left, the engine 565 generatescontent for the HMD 505 that mirrors the user's movement in a virtualenvironment or in an environment augmenting the local area withadditional content. Additionally, the engine 565 performs an actionwithin an application executing on the console 510 in response to anaction request received from the I/O interface 515 and provides feedbackto the user that the action was performed. The provided feedback may bevisual or audible feedback via the HMD 505 or haptic feedback via theI/O interface 515.

In some embodiments, based on the eye tracking information (e.g.,orientation of the user's eye) received from the eye tracking system545, the engine 565 determines resolution of the content provided to theHMD 505 for presentation to the user on the electronic display 525. Theengine 565 provides the content to the HMD 605 having a maximum pixelresolution on the electronic display 525 in a foveal region of theuser's gaze, whereas the engine 565 provides a lower pixel resolution inother regions of the electronic display 525, thus achieving less powerconsumption at the HMD 505 and saving computing cycles of the console510 without compromising a visual experience of the user. In someembodiments, the engine 565 can further use the eye tracking informationto adjust where objects are displayed on the electronic display 525 toprevent vergence-accommodation conflict.

ADDITIONAL CONFIGURATION INFORMATION

The foregoing description of the embodiments of the disclosure has beenpresented for the purpose of illustration; it is not intended to beexhaustive or to limit the disclosure to the precise forms disclosed.Persons skilled in the relevant art can appreciate that manymodifications and variations are possible in light of the abovedisclosure.

Some portions of this description describe the embodiments of thedisclosure in terms of algorithms and symbolic representations ofoperations on information. These algorithmic descriptions andrepresentations are commonly used by those skilled in the dataprocessing arts to convey the substance of their work effectively toothers skilled in the art. These operations, while describedfunctionally, computationally, or logically, are understood to beimplemented by computer programs or equivalent electrical circuits,microcode, or the like. Furthermore, it has also proven convenient attimes, to refer to these arrangements of operations as modules, withoutloss of generality. The described operations and their associatedmodules may be embodied in software, firmware, hardware, or anycombinations thereof.

Any of the steps, operations, or processes described herein may beperformed or implemented with one or more hardware or software modules,alone or in combination with other devices. In one embodiment, asoftware module is implemented with a computer program productcomprising a computer-readable medium containing computer program code,which can be executed by a computer processor for performing any or allof the steps, operations, or processes described.

Embodiments of the disclosure may also relate to an apparatus forperforming the operations herein. This apparatus may be speciallyconstructed for the required purposes, and/or it may comprise ageneral-purpose computing device selectively activated or reconfiguredby a computer program stored in the computer. Such a computer programmay be stored in a non-transitory, tangible computer readable storagemedium, or any type of media suitable for storing electronicinstructions, which may be coupled to a computer system bus.Furthermore, any computing systems referred to in the specification mayinclude a single processor or may be architectures employing multipleprocessor designs for increased computing capability.

Embodiments of the disclosure may also relate to a product that isproduced by a computing process described herein. Such a product maycomprise information resulting from a computing process, where theinformation is stored on a non-transitory, tangible computer readablestorage medium and may include any embodiment of a computer programproduct or other data combination described herein.

Finally, the language used in the specification has been principallyselected for readability and instructional purposes, and it may not havebeen selected to delineate or circumscribe the inventive subject matter.It is therefore intended that the scope of the disclosure be limited notby this detailed description, but rather by any claims that issue on anapplication based hereon. Accordingly, the disclosure of the embodimentsis intended to be illustrative, but not limiting, of the scope of thedisclosure, which is set forth in the following claims.

What is claimed is:
 1. A depth camera assembly (DCA) comprising: anillumination source configured to emit one or more optical beams; anacousto-optic deflector (AOD) that diffracts the one or more opticalbeams by at least one first diffraction angle to form diffractedscanning beams and adjusts the at least one first diffraction angle overa time period; a liquid crystal device (LCD) in an optical series withthe AOD, the LCD configured to diffract the diffracted scanning beamsfrom the AOD by at least one second diffraction angle larger than the atleast one first diffraction angle to generate structured light projectedinto a local area, the structured light covering a scanning field of thelocal area over the time period by scanning each sub-zone of thescanning field at a corresponding time instant of the time period basedon the adjustment of the at least one first diffraction angle and thediffraction by the at least one second diffraction angle; and an imagingdevice configured to capture one or more images of at least a portion ofthe structured light reflected in the local area.
 2. The DCA of claim 1,wherein the AOD includes at least one transducer and at least onediffraction area, and responsive to at least one radio frequency, the atleast one transducer is configured to generate at least one sound wavein the at least one diffraction area to form at least one dynamicdiffraction grating.
 3. The DCA of claim 2, further comprising: acontroller configured to modify the at least one radio frequency toadjust the at least one first diffraction angle at which the one or moreoptical beams are diffracted to form the diffracted scanning beams. 4.The DCA of claim 1, further comprising: a controller configured todetermine depth information for the local area based at least in part onthe captured one or more images.
 5. The DCA of claim 1, wherein: the AODcomprises a pair of AOD plates having mutually orthogonal orientationsof crystals in transducers, the pair of AOD plates generates thediffracted scanning beams in two dimensions, and each AOD plate isconfigured to diffract incident light by one diffraction angle of the atleast one first diffraction angle along one dimension.
 6. The DCA ofclaim 1, further comprising: a controller configured to instruct the AODto scan the local area at a set of diffraction angles over the timeperiod.
 7. The DCA of claim 1, wherein the LCD comprises a plurality ofliquid crystal gratings (LCGs), each LCG having an active state in whichthe LCG is configured to diffract the diffracted scanning beams from theAOD to generate the structured light.
 8. The DCA of claim 7, furthercomprising: a controller configured to control at most two of the LCGsto be in active states during the time period, wherein each LCG when inthe active state is configured to diffract incident light by a fixeddiffraction angle over at least a portion of the time period.
 9. The DCAof claim 1, wherein the imaging device includes: a camera; and apolarizing element positioned in front of the camera, the polarizingelement configured to propagate the portions of the reflected structuredlight having a polarization orthogonal to a polarization of thestructured light.
 10. The DCA of claim 9, further comprising: acontroller configured to determine depth information for the local areabased in part on the polarization of the reflected structured light. 11.A method comprising: generating emission instructions; providing theemission instructions to an acousto-optic deflector (AOD) to generatediffracted scanning beams from one or more optical beams by diffractingthe one or more optical beams from the AOD by at least one firstdiffraction angle and by adjusting the at least one first diffractionangle over a time period based in part on the emission instructions toform the diffracted scanning beams; providing the emission instructionsto a liquid crystal device (LCD) in an optical series with the AOD togenerate structured light projected into a local area by diffracting thediffracted scanning beams by at least one second diffraction anglelarger than the at least one first diffraction angle based in part onthe emission instructions, the structured light covering a scanningfield of the local area over the time period by scanning each sub-zoneof the scanning field at a corresponding time instant of the time periodbased on the adjustment of the at least one first diffraction angle andthe diffraction by the at least one second diffraction angle; andcapturing one or more images of at least a portion of the structuredlight reflected in the local area.
 12. The method of claim 11, furthercomprising: creating the emission instructions comprising at least oneradio frequency at which the AOD is driven; providing the emissioninstructions comprising the at least one radio frequency to the AODhaving at least one transducer and at least one diffraction area; andgenerating, by the at least one transducer responsive to the at leastone radio frequency in the emission instructions, at least one soundwave in the at least one diffraction area to form at least one dynamicdiffraction grating that diffracts the one or more optical beams by theat least one first diffraction angle.
 13. The method of claim 12,further comprising: modifying the at least one radio frequency to adjustthe at least one first diffraction angle at which the one or moreoptical beams are diffracted to form the diffracted scanning beams. 14.The method of claim 11, further comprising: determining depthinformation for the local area based at least in part on the capturedone or more images.
 15. The method of claim 11, wherein the LCDcomprises a plurality of liquid crystal gratings (LCGs), each LCG havingan active state in which the LCG is configured to diffract thediffracted scanning beams from the AOD to generate the structured light.16. The method of claim 15, further comprising: controlling at most twoof the LCGs to be in active states during the time period based in parton the emission instructions, each LCG when in the active statediffracting incident light by a fixed diffraction angle over at least aportion of the time period.
 17. The method of claim 11, furthercomprising: controlling the AOD to scan the local area at a set ofdiffraction angles over the time period, based in part on the emissioninstructions.
 18. The method of claim 11, further comprising:propagating the portion of the reflected structured light having apolarization orthogonal to a polarization of the structured light; anddetermining depth information for the local area based in part on thepolarization of the reflected structured light.
 19. A structured lightgenerator comprising: an illumination source configured to emit one ormore optical beams; an acousto-optic deflector (AOD) that diffracts theone or more optical beams by at least one first diffraction angle toform diffracted scanning beams and adjusts the at least one firstdiffraction angle over a time period; and a liquid crystal device (LCD)in an optical series with the AOD, the LCD configured to diffract thediffracted scanning beams from the AOD by at least one seconddiffraction angle larger than the at least one first diffraction angleto generate structured light projected into a local area, the structuredlight covering a scanning field of the local area over the time periodby scanning each sub-zone of the scanning field at a corresponding timeinstant of the time period based on the adjustment of the at least onefirst diffraction angle and the diffraction by the at least one seconddiffraction angle.
 20. The structured light generator of claim 19,wherein the AOD includes at least one transducer and at least onediffraction area, and responsive to at least one radio frequency, the atleast one transducer is configured to generate at least one sound wavein the at least one diffraction area to form at least one dynamicdiffraction grating.